often you canât tell whether someone is Jewish just by looking at them
That is relatively incorrect and is a matter of perspective. Religious Jews are recognizable by the way they dress, most people when they think of Jewish, think of yarmulkes or the brimmed hat or the payot (curled or hairs on the cheek).
There are Jews that are secular that don't keep to those, but even then, in many cases when it comes to attacks on Jewish people (primarily by people of African origin) it's on the religious Jews.
However even if you are a "secular" person of Jewish origin, people from the Middle East that grow up around Israel, know how to recognize visual traits of people with Jewish origin, it's a lot less precise given humans tend to mix, but people in Europe can often recognize people from nearby countries by certain physical visual traits as well. Eg. you can recognize Dutch because they are tall with an "elongated" head, Germans have straight nose, square jaws and look "rounder".
People in the West have trouble distinguishing Chinese from Korean, people in Asia however can recognize different parts of either country which is the most famous example of how people can pick up traits without ever being taught what any of it means by their parents.
Same goes for black people, once you are around them long enough, you can find people from Creole (eg. Jamaican) backgrounds, actual Africans and American blacks are VERY different, and I'm sure if someone lives in Africa, they can recognize tribal features as well. Even amongst American blacks, you can often recognize black people historically living in the Northerner parts of America (lighter, more features of Native Americans) vs Southern black families.
